{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Getting language models to use external tools (a.k.a. function calls)\n",
    "\n",
    "**Note that this notebook is a work in progress, and is not in its final article-style form. But it is already a working introduction to tool use and function calling in guidance.**\n",
    "\n",
    "This notebook demonstrates how to get LLMs to call external tools when needed. Tools use can be implementated many ways, because there are many possible ways to design prompts that then produce outputs that can be parsed to trigger external tool calls. You can create and parse of all this syntax yourself, but Guidance also has special support commands for this that align with both the way LLMs are actually executed and with popular APIs like OpenAI. Using this syntax will also help ensure you align your prompts with any fine-tuning the LLM may have undergone for tool use (assuming the corresponding LLM object in Guidance has support built in)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## OpenAI Chat Models\n",
    "\n",
    "Tool use in Guidance is designed to align with how the model actually processes the text you give it. This means you give the model the actual function definition text the model sees, and you watch for the text generated by the model when it wants to make a function call. While the OpenAI Chat API abstracts away all these details, Guidance re-exposes them so you can interact with OpenAI models in the same way you would interact with any other model.\n",
    "\n",
    "So in the examples below you will see text going into the model's system prompt, and function calls coming out of the model as though you were watching the raw model output inside the `assistant` role. But behind the scenes the `guidance.llms.OpenAI` class translates this text into the corrresponding API calls. Note that the text that Guidance puts into the system prompt follows the TypeScript format that ChatGPT claims to expect on the backend, so you are seeing what things look like to the LLM itself (we just asked ChatGPT what it expects in order to get this format)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the tool(s) we want to use\n",
    "\n",
    "Here we use the same mock tool that is used in the OpenAI docs, a mock weather service function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define a tool we would like the model to use\n",
    "import json\n",
    "def get_current_weather(location, unit=\"fahrenheit\"):\n",
    "    \"\"\" Get the current weather in a given location.\n",
    "    \n",
    "    Parameters\n",
    "    ----------\n",
    "    location : string\n",
    "        The city and state, e.g. San Francisco, CA\n",
    "    unit : \"celsius\" or \"fahrenheit\"\n",
    "    \"\"\"\n",
    "    weather_info = {\n",
    "        \"location\": location,\n",
    "        \"temperature\": \"71\",\n",
    "        \"unit\": unit,\n",
    "        \"forecast\": [\"sunny\", \"windy\"],\n",
    "    }\n",
    "    return json.dumps(weather_info)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define a guidance program that uses the tool(s)\n",
    "\n",
    "To get the LLM to use tools when it needs to you need to first specify which tools it can use, then you need to watch for when the LLM wants to use a tool.\n",
    "\n",
    "**Function definition**: There are many ways you can tell the LLM about functions it can use, but in Guidance by convention we use the `tool_def` partial to write this definition. Each LLM object defines `tool_def` so that you can use it and know that if your model was fine tuned for tool use you will align with how the model was trained. The `tool_def` program is (normally) defined by the LLM running your program and it will convert a list of function definitions into a format that the model understands (where the function definitions are the same as the OpenAI API expects). For OpenAI models the text generated by `tool_def` is TypeScript type definitions and belongs at the end of the `system` message. *Note: Any variables not found in the current program scope fall back to the LLM object, so writing `tool_def` falls back to `llm.tool_def` unless you have explicitly defined a custom `tool_def` variable in the current program.*\n",
    "\n",
    "**Call detection:** Calls to functions by the LLM can be detected manually by setting the `stop` or `stop_regex` parameters of the `gen` command to something that signifies that the LLM is making a function call. But a cleaner way is to use the `function_call=\"auto\"` parameter. This will get passed directly to the LLM object so that it can set the appropriate `stop_regex` parameter or API parameter (to change how this work you can override the `function_call_stop` or `function_call_stop_regex` variables). There is also a `extract_function_call` variable that allows you to extract a callable object from the text returned by `gen` calls. Rather that calling this manually you can also treat the returned text just like a function and Guidance will use the `extract_function_call` command in the background, so calling the string will result in calling the tool call embedded in that string. This makes it easy to work with tool call outputs in the same way you work with other outputs from the LLM.\n",
    "\n",
    "In summary there are four special variables and one `gen` argument that are used to implement tool use in Guidance. All of them have default implementations defined by the LLM object, but you can override them to change how tool use works:\n",
    "\n",
    "- `tool_def`: A guidance program that defines the tool(s) that the LLM can use, it looks for a `functions` variable that has function definitions in the OpenAI dictionary-style function definition syntax.\n",
    "- `function_call`: This parameter of the `gen` command is passed directly to the LLM object to tell it if it should generate function calls.\n",
    "- `extract_function_call`: A function that takes the text returned by the LLM and extracts a callable object from it.\n",
    "- `function_call_stop`: A string that is used to detect when the LLM is making a function call.\n",
    "- `function_call_stop_regex`: A regex that is used to detect when the LLM is making a function call.\n",
    "\n",
    "\n",
    "Below is an example that puts all this together for the OpenAI Chat API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import guidance\n",
    "\n",
    "# define the chat model we want to use (must be a recent model supporting function calls)\n",
    "guidance.llm = guidance.llms.OpenAI(\"gpt-3.5-turbo-0613\", caching=False)\n",
    "\n",
    "# define a guidance program that uses tools\n",
    "program = guidance(\"\"\"\n",
    "{{~#system~}}\n",
    "You are a helpful assistant.\n",
    "{{>tool_def functions=functions}}\n",
    "{{~/system~}}\n",
    "\n",
    "{{~#user~}}\n",
    "Get the current weather in New York City.\n",
    "{{~/user~}}\n",
    "\n",
    "{{~#each range(10)~}}\n",
    "    {{~#assistant~}}\n",
    "    {{gen 'answer' max_tokens=50 function_call=\"auto\"}}\n",
    "    {{~/assistant~}}\n",
    "\n",
    "    {{#if not callable(answer)}}{{break}}{{/if}}\n",
    "    \n",
    "    {{~#function name=answer.__name__~}}\n",
    "    {{answer()}}\n",
    "    {{~/function~}}\n",
    "{{~/each~}}\"\"\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that in the program above we make a maximum of 10 consecutive function calls. Once the model generates text that does not include a function call we break out and leave the text from that final answer in the `answer` variable.\n",
    "\n",
    "### Calling the guidance program\n",
    "\n",
    "To call the program above we need to pass in a function definition and the actual function to call."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div id=\"guidance-stop-button-6ad8e50d-1a62-4325-8a49-ec3eeb12df5f\" style=\"cursor: pointer; margin: 0px; display: none; float: right; padding: 3px; border-radius: 4px 4px 4px 4px; border: 0px solid rgba(127, 127, 127, 1); padding-left: 10px; padding-right: 10px; font-size: 13px; background-color: rgba(127, 127, 127, 0.25);\">Stop program</div><div id=\"guidance-content-6ad8e50d-1a62-4325-8a49-ec3eeb12df5f\"><pre style='margin: 0px; padding: 0px; padding-left: 8px; margin-left: -8px; border-radius: 0px; border-left: 1px solid rgba(127, 127, 127, 0.2); white-space: pre-wrap; font-family: ColfaxAI, Arial; font-size: 15px; line-height: 23px;'><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>system</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'>You are a helpful assistant.\n",
       "\n",
       "# Tools\n",
       "\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if len(functions) &gt; 0~}}\n",
       "## functions\n",
       "\n",
       "namespace functions {\n",
       "\n",
       "{{#each functions item_name=&quot;function&quot;~}}\n",
       "// {{function.description}}\n",
       "type {{function.name}} = (_: {\n",
       "{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}\n",
       "}) =&gt; any;\n",
       "\n",
       "{{/each~}}\n",
       "} // namespace functions\n",
       "{{~/if~}}'>## functions\n",
       "\n",
       "namespace functions {\n",
       "\n",
       "<span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{#each functions item_name=&quot;function&quot;~}}\n",
       "// {{function.description}}\n",
       "type {{function.name}} = (_: {\n",
       "{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}\n",
       "}) =&gt; any;\n",
       "\n",
       "{{/each~}}\n",
       "'>// <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{function.description}}'>Get the current weather in a given location</span>\n",
       "type <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{function.name}}'>get_current_weather</span> = (_: <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{<span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}'></span>\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "'>// <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this.description}}'>The city and state, e.g. San Francisco, CA</span>\n",
       "</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{@key}}'>location</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless contains(function.parameters.required, @key)}}?{{/unless}}'></span>: <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}'><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this.type}}'>string</span></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}},{{/unless}}'>,</span>\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "'></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{@key}}'>unit</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless contains(function.parameters.required, @key)}}?{{/unless}}'>?</span>: <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}'><span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}'>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this}}'>celsius</span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}} | {{/unless}}'> | </span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this}}'>fahrenheit</span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}} | {{/unless}}'></span></span></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}},{{/unless}}'></span></span>\n",
       "}) =&gt; any;\n",
       "\n",
       "</span>} // namespace functions</span></div></div><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>user</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'>Get the current weather in New York City.</div></div><span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{~#each range(10)~}}\n",
       "    {{~#assistant~}}\n",
       "    {{gen &#x27;answer&#x27; max_tokens=50 function_call=&quot;auto&quot;}}\n",
       "    {{~/assistant~}}\n",
       "\n",
       "    {{#if not callable(answer)}}{{break}}{{/if}}\n",
       "    \n",
       "    {{~#function name=answer.__name__~}}\n",
       "    {{answer()}}\n",
       "    {{~/function~}}\n",
       "{{~/each~}}'><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>assistant</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'><span style='background-color: rgba(0, 165, 0, 0.25); opacity: 1.0; display: inline;' title='{{gen &#x27;answer&#x27; max_tokens=50 function_call=&quot;auto&quot;}}'>\n",
       "```typescript\n",
       "functions.get_current_weather({\n",
       "  &quot;location&quot;: &quot;New York City&quot;\n",
       "})```</span></div></div><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if not callable(answer)}}{{break}}{{/if}}'></span><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>function</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{answer()}}'>{&quot;location&quot;: &quot;New York City&quot;, &quot;temperature&quot;: &quot;71&quot;, &quot;unit&quot;: &quot;fahrenheit&quot;, &quot;forecast&quot;: [&quot;sunny&quot;, &quot;windy&quot;]}</span></div></div><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>assistant</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'><span style='background-color: rgba(0, 165, 0, 0.25); opacity: 1.0; display: inline;' title='{{gen &#x27;answer&#x27; max_tokens=50 function_call=&quot;auto&quot;}}'>The current weather in New York City is 71°F and it is sunny and windy.</span></div></div><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if not callable(answer)}}{{break}}{{/if}}'><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{break}}'></span></span></span></pre></div>\n",
       "<script type=\"text/javascript\">(()=>{var t={296:(t,e,n)=>{var i=NaN,o=\"[object Symbol]\",r=/^\\s+|\\s+$/g,a=/^[-+]0x[0-9a-f]+$/i,s=/^0b[01]+$/i,c=/^0o[0-7]+$/i,d=parseInt,u=\"object\"==typeof n.g&&n.g&&n.g.Object===Object&&n.g,l=\"object\"==typeof self&&self&&self.Object===Object&&self,f=u||l||Function(\"return this\")(),h=Object.prototype.toString,p=Math.max,m=Math.min,g=function(){return f.Date.now()};function b(t){var e=typeof t;return!!t&&(\"object\"==e||\"function\"==e)}function y(t){if(\"number\"==typeof t)return t;if(function(t){return\"symbol\"==typeof t||function(t){return!!t&&\"object\"==typeof t}(t)&&h.call(t)==o}(t))return i;if(b(t)){var e=\"function\"==typeof t.valueOf?t.valueOf():t;t=b(e)?e+\"\":e}if(\"string\"!=typeof t)return 0===t?t:+t;t=t.replace(r,\"\");var n=s.test(t);return n||c.test(t)?d(t.slice(2),n?2:8):a.test(t)?i:+t}t.exports=function(t,e,n){var i,o,r,a,s,c,d=0,u=!1,l=!1,f=!0;if(\"function\"!=typeof t)throw new TypeError(\"Expected a function\");function h(e){var n=i,r=o;return i=o=void 0,d=e,a=t.apply(r,n)}function v(t){var n=t-c;return void 0===c||n>=e||n<0||l&&t-d>=r}function _(){var t=g();if(v(t))return w(t);s=setTimeout(_,function(t){var n=e-(t-c);return l?m(n,r-(t-d)):n}(t))}function w(t){return s=void 0,f&&i?h(t):(i=o=void 0,a)}function j(){var t=g(),n=v(t);if(i=arguments,o=this,c=t,n){if(void 0===s)return function(t){return d=t,s=setTimeout(_,e),u?h(t):a}(c);if(l)return s=setTimeout(_,e),h(c)}return void 0===s&&(s=setTimeout(_,e)),a}return e=y(e)||0,b(n)&&(u=!!n.leading,r=(l=\"maxWait\"in n)?p(y(n.maxWait)||0,e):r,f=\"trailing\"in n?!!n.trailing:f),j.cancel=function(){void 0!==s&&clearTimeout(s),d=0,i=c=o=s=void 0},j.flush=function(){return void 0===s?a:w(g())},j}},777:t=>{var e,n,i=Math.max,o=(e=function(t,e){return function(t,e,n){if(\"function\"!=typeof t)throw new TypeError(\"Expected a function\");return setTimeout((function(){t.apply(void 0,n)}),1)}(t,0,e)},n=i(void 0===n?e.length-1:n,0),function(){for(var t=arguments,o=-1,r=i(t.length-n,0),a=Array(r);++o<r;)a[o]=t[n+o];o=-1;for(var s=Array(n+1);++o<n;)s[o]=t[o];return s[n]=a,function(t,e,n){switch(n.length){case 0:return t.call(e);case 1:return t.call(e,n[0]);case 2:return t.call(e,n[0],n[1]);case 3:return t.call(e,n[0],n[1],n[2])}return t.apply(e,n)}(e,this,s)});t.exports=o}},e={};function n(i){var o=e[i];if(void 0!==o)return o.exports;var r=e[i]={exports:{}};return t[i](r,r.exports,n),r.exports}n.n=t=>{var e=t&&t.__esModule?()=>t.default:()=>t;return n.d(e,{a:e}),e},n.d=(t,e)=>{for(var i in e)n.o(e,i)&&!n.o(t,i)&&Object.defineProperty(t,i,{enumerable:!0,get:e[i]})},n.g=function(){if(\"object\"==typeof globalThis)return globalThis;try{return this||new Function(\"return this\")()}catch(t){if(\"object\"==typeof window)return window}}(),n.o=(t,e)=>Object.prototype.hasOwnProperty.call(t,e),(()=>{\"use strict\";const t=t=>{const e=new Set;do{for(const n of Reflect.ownKeys(t))e.add([t,n])}while((t=Reflect.getPrototypeOf(t))&&t!==Object.prototype);return e};function e(e,{include:n,exclude:i}={}){const o=t=>{const e=e=>\"string\"==typeof e?t===e:e.test(t);return n?n.some(e):!i||!i.some(e)};for(const[n,i]of t(e.constructor.prototype)){if(\"constructor\"===i||!o(i))continue;const t=Reflect.getOwnPropertyDescriptor(n,i);t&&\"function\"==typeof t.value&&(e[i]=e[i].bind(e))}return e}var i=n(777),o=n.n(i),r=n(296),a=n.n(r);class s{constructor(t,n){e(this),this.interfaceId=t,this.callbackMap={},this.data={},this.pendingData={},this.jcomm=new c(\"guidance_interface_target_\"+this.interfaceId,this.updateData,\"open\"),this.debouncedSendPendingData500=a()(this.sendPendingData,500),this.debouncedSendPendingData1000=a()(this.sendPendingData,1e3),n&&o()(n)}send(t,e){this.addPendingData(t,e),this.sendPendingData()}sendEvent(t){for(const e of Object.keys(t))this.addPendingData(e,t[e]);this.sendPendingData()}debouncedSendEvent500(t){for(const e of Object.keys(t))this.addPendingData(e,t[e]);this.debouncedSendPendingData500()}debouncedSend500(t,e){this.addPendingData(t,e),this.debouncedSendPendingData500()}debouncedSend1000(t,e){this.addPendingData(t,e),this.debouncedSendPendingData1000()}addPendingData(t,e){Array.isArray(t)||(t=[t]);for(const n in t)this.pendingData[t[n]]=e}updateData(t){t=JSON.parse(t.data);for(const e in t)this.data[e]=t[e];for(const e in t)e in this.callbackMap&&this.callbackMap[e](this.data[e])}subscribe(t,e){this.callbackMap[t]=e,o()((e=>this.callbackMap[t](this.data[t])))}sendPendingData(){this.jcomm.send_data(this.pendingData),this.pendingData={}}}class c{constructor(t,e,n=\"open\"){this._fire_callback=this._fire_callback.bind(this),this._register=this._register.bind(this),this.jcomm=void 0,this.callback=e,void 0!==window.Jupyter?\"register\"===n?Jupyter.notebook.kernel.comm_manager.register_target(t,this._register):(this.jcomm=Jupyter.notebook.kernel.comm_manager.new_comm(t),this.jcomm.on_msg(this._fire_callback)):void 0!==window._mgr&&(\"register\"===n?window._mgr.widgetManager.proxyKernel.registerCommTarget(t,this._register):(this.jcomm=window._mgr.widgetManager.proxyKernel.createComm(t),this.jcomm.open({},\"\"),this.jcomm.onMsg=this._fire_callback))}send_data(t){void 0!==this.jcomm?this.jcomm.send(t):console.error(\"Jupyter comm module not yet loaded! So we can't send the message.\")}_register(t,e){this.jcomm=t,this.jcomm.on_msg(this._fire_callback)}_fire_callback(t){this.callback(t.content.data)}}class d{constructor(t,n){e(this),this.id=t,this.comm=new s(t),this.comm.subscribe(\"append\",this.appendData),this.comm.subscribe(\"replace\",this.replaceData),this.comm.subscribe(\"event\",this.eventOccurred),this.element=document.getElementById(\"guidance-content-\"+t),this.stop_button=document.getElementById(\"guidance-stop-button-\"+t),this.stop_button.onclick=()=>this.comm.send(\"event\",\"stop\")}appendData(t){t&&(this.stop_button.style.display=\"inline-block\",this.element.innerHTML+=t)}replaceData(t){t&&(this.stop_button.style.display=\"inline-block\",this.element.innerHTML=t)}eventOccurred(t){\"complete\"===t&&(this.stop_button.style.display=\"none\")}}window._guidanceDisplay=function(t,e){return new d(t,e)}})()})();; window._guidanceDisplay(\"6ad8e50d-1a62-4325-8a49-ec3eeb12df5f\");</script>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# call the program, passing in the function definition we want to use as JSON\n",
    "executed_program = program(functions=[\n",
    "    {\n",
    "        \"name\": \"get_current_weather\",\n",
    "        \"description\": \"Get the current weather in a given location\",\n",
    "        \"parameters\": {\n",
    "            \"type\": \"object\",\n",
    "            \"properties\": {\n",
    "                \"location\": {\n",
    "                    \"type\": \"string\",\n",
    "                    \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
    "                },\n",
    "                \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n",
    "            },\n",
    "            \"required\": [\"location\"],\n",
    "        }\n",
    "    }\n",
    "], get_current_weather=get_current_weather)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The current weather in New York City is 71°F and it is sunny and windy.'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "executed_program[\"answer\"]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Factoring out the loop calling code\n",
    "\n",
    "There is a non-trivial amount of logic and syntax required to create a loop of function calls. We can make this easer by factoring out that loop into its own guidance program:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div id=\"guidance-stop-button-afc32bba-76bf-4676-88b7-82826d000548\" style=\"cursor: pointer; margin: 0px; display: none; float: right; padding: 3px; border-radius: 4px 4px 4px 4px; border: 0px solid rgba(127, 127, 127, 1); padding-left: 10px; padding-right: 10px; font-size: 13px; background-color: rgba(127, 127, 127, 0.25);\">Stop program</div><div id=\"guidance-content-afc32bba-76bf-4676-88b7-82826d000548\"><pre style='margin: 0px; padding: 0px; padding-left: 8px; margin-left: -8px; border-radius: 0px; border-left: 1px solid rgba(127, 127, 127, 0.2); white-space: pre-wrap; font-family: ColfaxAI, Arial; font-size: 15px; line-height: 23px;'><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>system</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'>You are a helpful assistant.\n",
       "\n",
       "# Tools\n",
       "\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if len(functions) &gt; 0~}}\n",
       "## functions\n",
       "\n",
       "namespace functions {\n",
       "\n",
       "{{#each functions item_name=&quot;function&quot;~}}\n",
       "// {{function.description}}\n",
       "type {{function.name}} = (_: {\n",
       "{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}\n",
       "}) =&gt; any;\n",
       "\n",
       "{{/each~}}\n",
       "} // namespace functions\n",
       "{{~/if~}}'>## functions\n",
       "\n",
       "namespace functions {\n",
       "\n",
       "<span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{#each functions item_name=&quot;function&quot;~}}\n",
       "// {{function.description}}\n",
       "type {{function.name}} = (_: {\n",
       "{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}\n",
       "}) =&gt; any;\n",
       "\n",
       "{{/each~}}\n",
       "'>// <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{function.description}}'>Get the current weather in a given location</span>\n",
       "type <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{function.name}}'>get_current_weather</span> = (_: <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{<span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}'></span>\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "'>// <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this.description}}'>The city and state, e.g. San Francisco, CA</span>\n",
       "</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{@key}}'>location</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless contains(function.parameters.required, @key)}}?{{/unless}}'></span>: <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}'><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this.type}}'>string</span></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}},{{/unless}}'>,</span>\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "'></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{@key}}'>unit</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless contains(function.parameters.required, @key)}}?{{/unless}}'>?</span>: <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}'><span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}'>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this}}'>celsius</span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}} | {{/unless}}'> | </span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this}}'>fahrenheit</span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}} | {{/unless}}'></span></span></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}},{{/unless}}'></span></span>\n",
       "}) =&gt; any;\n",
       "\n",
       "</span>} // namespace functions</span></div></div><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>user</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'>Get the current weather in New York City.</div></div><span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{~#each range(max_calls)~}}\n",
       "    {{~#assistant~}}\n",
       "    {{gen &#x27;func_inner&#x27; temperature=temperature max_tokens=max_tokens_per_chunk function_call=function_call~}}\n",
       "    {{~/assistant~}}\n",
       "\n",
       "    {{#if not callable(func_inner)}}{{break}}{{/if}}\n",
       "\n",
       "    {{~#function name=func_inner.__name__~}}\n",
       "    {{func_inner()}}\n",
       "    {{~/function~}}\n",
       "{{~/each~}}'><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>assistant</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'><span style='background-color: rgba(0, 165, 0, 0.25); opacity: 1.0; display: inline;' title='{{gen &#x27;func_inner&#x27; temperature=temperature max_tokens=max_tokens_per_chunk function_call=function_call~}}\n",
       "    '>\n",
       "```typescript\n",
       "functions.get_current_weather({\n",
       "  &quot;location&quot;: &quot;New York City&quot;\n",
       "})```</span></div></div><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if not callable(func_inner)}}{{break}}{{/if}}'></span><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>function</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{func_inner()}}'>{&quot;location&quot;: &quot;New York City&quot;, &quot;temperature&quot;: &quot;71&quot;, &quot;unit&quot;: &quot;fahrenheit&quot;, &quot;forecast&quot;: [&quot;sunny&quot;, &quot;windy&quot;]}</span></div></div><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>assistant</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'><span style='background-color: rgba(0, 165, 0, 0.25); opacity: 1.0; display: inline;' title='{{gen &#x27;func_inner&#x27; temperature=temperature max_tokens=max_tokens_per_chunk function_call=function_call~}}\n",
       "    '>The current weather in New York City is 71°F and sunny with windy conditions.</span></div></div><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if not callable(func_inner)}}{{break}}{{/if}}'><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{break}}'></span></span></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{set args[0] func_inner}}'></span></pre></div>\n",
       "<script type=\"text/javascript\">(()=>{var t={296:(t,e,n)=>{var i=NaN,o=\"[object Symbol]\",r=/^\\s+|\\s+$/g,a=/^[-+]0x[0-9a-f]+$/i,s=/^0b[01]+$/i,c=/^0o[0-7]+$/i,d=parseInt,u=\"object\"==typeof n.g&&n.g&&n.g.Object===Object&&n.g,l=\"object\"==typeof self&&self&&self.Object===Object&&self,f=u||l||Function(\"return this\")(),h=Object.prototype.toString,p=Math.max,m=Math.min,g=function(){return f.Date.now()};function b(t){var e=typeof t;return!!t&&(\"object\"==e||\"function\"==e)}function y(t){if(\"number\"==typeof t)return t;if(function(t){return\"symbol\"==typeof t||function(t){return!!t&&\"object\"==typeof t}(t)&&h.call(t)==o}(t))return i;if(b(t)){var e=\"function\"==typeof t.valueOf?t.valueOf():t;t=b(e)?e+\"\":e}if(\"string\"!=typeof t)return 0===t?t:+t;t=t.replace(r,\"\");var n=s.test(t);return n||c.test(t)?d(t.slice(2),n?2:8):a.test(t)?i:+t}t.exports=function(t,e,n){var i,o,r,a,s,c,d=0,u=!1,l=!1,f=!0;if(\"function\"!=typeof t)throw new TypeError(\"Expected a function\");function h(e){var n=i,r=o;return i=o=void 0,d=e,a=t.apply(r,n)}function v(t){var n=t-c;return void 0===c||n>=e||n<0||l&&t-d>=r}function _(){var t=g();if(v(t))return w(t);s=setTimeout(_,function(t){var n=e-(t-c);return l?m(n,r-(t-d)):n}(t))}function w(t){return s=void 0,f&&i?h(t):(i=o=void 0,a)}function j(){var t=g(),n=v(t);if(i=arguments,o=this,c=t,n){if(void 0===s)return function(t){return d=t,s=setTimeout(_,e),u?h(t):a}(c);if(l)return s=setTimeout(_,e),h(c)}return void 0===s&&(s=setTimeout(_,e)),a}return e=y(e)||0,b(n)&&(u=!!n.leading,r=(l=\"maxWait\"in n)?p(y(n.maxWait)||0,e):r,f=\"trailing\"in n?!!n.trailing:f),j.cancel=function(){void 0!==s&&clearTimeout(s),d=0,i=c=o=s=void 0},j.flush=function(){return void 0===s?a:w(g())},j}},777:t=>{var e,n,i=Math.max,o=(e=function(t,e){return function(t,e,n){if(\"function\"!=typeof t)throw new TypeError(\"Expected a function\");return setTimeout((function(){t.apply(void 0,n)}),1)}(t,0,e)},n=i(void 0===n?e.length-1:n,0),function(){for(var t=arguments,o=-1,r=i(t.length-n,0),a=Array(r);++o<r;)a[o]=t[n+o];o=-1;for(var s=Array(n+1);++o<n;)s[o]=t[o];return s[n]=a,function(t,e,n){switch(n.length){case 0:return t.call(e);case 1:return t.call(e,n[0]);case 2:return t.call(e,n[0],n[1]);case 3:return t.call(e,n[0],n[1],n[2])}return t.apply(e,n)}(e,this,s)});t.exports=o}},e={};function n(i){var o=e[i];if(void 0!==o)return o.exports;var r=e[i]={exports:{}};return t[i](r,r.exports,n),r.exports}n.n=t=>{var e=t&&t.__esModule?()=>t.default:()=>t;return n.d(e,{a:e}),e},n.d=(t,e)=>{for(var i in e)n.o(e,i)&&!n.o(t,i)&&Object.defineProperty(t,i,{enumerable:!0,get:e[i]})},n.g=function(){if(\"object\"==typeof globalThis)return globalThis;try{return this||new Function(\"return this\")()}catch(t){if(\"object\"==typeof window)return window}}(),n.o=(t,e)=>Object.prototype.hasOwnProperty.call(t,e),(()=>{\"use strict\";const t=t=>{const e=new Set;do{for(const n of Reflect.ownKeys(t))e.add([t,n])}while((t=Reflect.getPrototypeOf(t))&&t!==Object.prototype);return e};function e(e,{include:n,exclude:i}={}){const o=t=>{const e=e=>\"string\"==typeof e?t===e:e.test(t);return n?n.some(e):!i||!i.some(e)};for(const[n,i]of t(e.constructor.prototype)){if(\"constructor\"===i||!o(i))continue;const t=Reflect.getOwnPropertyDescriptor(n,i);t&&\"function\"==typeof t.value&&(e[i]=e[i].bind(e))}return e}var i=n(777),o=n.n(i),r=n(296),a=n.n(r);class s{constructor(t,n){e(this),this.interfaceId=t,this.callbackMap={},this.data={},this.pendingData={},this.jcomm=new c(\"guidance_interface_target_\"+this.interfaceId,this.updateData,\"open\"),this.debouncedSendPendingData500=a()(this.sendPendingData,500),this.debouncedSendPendingData1000=a()(this.sendPendingData,1e3),n&&o()(n)}send(t,e){this.addPendingData(t,e),this.sendPendingData()}sendEvent(t){for(const e of Object.keys(t))this.addPendingData(e,t[e]);this.sendPendingData()}debouncedSendEvent500(t){for(const e of Object.keys(t))this.addPendingData(e,t[e]);this.debouncedSendPendingData500()}debouncedSend500(t,e){this.addPendingData(t,e),this.debouncedSendPendingData500()}debouncedSend1000(t,e){this.addPendingData(t,e),this.debouncedSendPendingData1000()}addPendingData(t,e){Array.isArray(t)||(t=[t]);for(const n in t)this.pendingData[t[n]]=e}updateData(t){t=JSON.parse(t.data);for(const e in t)this.data[e]=t[e];for(const e in t)e in this.callbackMap&&this.callbackMap[e](this.data[e])}subscribe(t,e){this.callbackMap[t]=e,o()((e=>this.callbackMap[t](this.data[t])))}sendPendingData(){this.jcomm.send_data(this.pendingData),this.pendingData={}}}class c{constructor(t,e,n=\"open\"){this._fire_callback=this._fire_callback.bind(this),this._register=this._register.bind(this),this.jcomm=void 0,this.callback=e,void 0!==window.Jupyter?\"register\"===n?Jupyter.notebook.kernel.comm_manager.register_target(t,this._register):(this.jcomm=Jupyter.notebook.kernel.comm_manager.new_comm(t),this.jcomm.on_msg(this._fire_callback)):void 0!==window._mgr&&(\"register\"===n?window._mgr.widgetManager.proxyKernel.registerCommTarget(t,this._register):(this.jcomm=window._mgr.widgetManager.proxyKernel.createComm(t),this.jcomm.open({},\"\"),this.jcomm.onMsg=this._fire_callback))}send_data(t){void 0!==this.jcomm?this.jcomm.send(t):console.error(\"Jupyter comm module not yet loaded! So we can't send the message.\")}_register(t,e){this.jcomm=t,this.jcomm.on_msg(this._fire_callback)}_fire_callback(t){this.callback(t.content.data)}}class d{constructor(t,n){e(this),this.id=t,this.comm=new s(t),this.comm.subscribe(\"append\",this.appendData),this.comm.subscribe(\"replace\",this.replaceData),this.comm.subscribe(\"event\",this.eventOccurred),this.element=document.getElementById(\"guidance-content-\"+t),this.stop_button=document.getElementById(\"guidance-stop-button-\"+t),this.stop_button.onclick=()=>this.comm.send(\"event\",\"stop\")}appendData(t){t&&(this.stop_button.style.display=\"inline-block\",this.element.innerHTML+=t)}replaceData(t){t&&(this.stop_button.style.display=\"inline-block\",this.element.innerHTML=t)}eventOccurred(t){\"complete\"===t&&(this.stop_button.style.display=\"none\")}}window._guidanceDisplay=function(t,e){return new d(t,e)}})()})();; window._guidanceDisplay(\"afc32bba-76bf-4676-88b7-82826d000548\");</script>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# this is a reusabe component for calling functions as intermediate steps in a generation\n",
    "# (note that args[0] refers to the first positional argument passed to the program when it is included)\n",
    "chat_tool_gen = guidance(\"\"\"{{~#each range(max_calls)~}}\n",
    "    {{~#assistant~}}\n",
    "    {{gen 'func_inner' temperature=temperature max_tokens=max_tokens_per_chunk function_call=function_call~}}\n",
    "    {{~/assistant~}}\n",
    "\n",
    "    {{#if not callable(func_inner)}}{{break}}{{/if}}\n",
    "\n",
    "    {{~#function name=func_inner.__name__~}}\n",
    "    {{func_inner()}}\n",
    "    {{~/function~}}\n",
    "{{~/each~}}{{set args[0] func_inner}}\"\"\", max_calls=20, function_call=\"auto\", max_tokens_per_chunk=500, temperature=0.0)\n",
    "\n",
    "# define a guidance program that uses chat_tool_gen\n",
    "program2 = guidance(\"\"\"\n",
    "{{~#system~}}\n",
    "You are a helpful assistant.\n",
    "{{>tool_def functions=functions}}\n",
    "{{~/system~}}\n",
    "\n",
    "{{~#user~}}\n",
    "Get the current weather in New York City.\n",
    "{{~/user~}}\n",
    "\n",
    "{{>chat_tool_gen 'answer'}}\"\"\", chat_tool_gen=chat_tool_gen)\n",
    "\n",
    "# call the program\n",
    "executed_program2 = program2(functions=[\n",
    "    {\n",
    "        \"name\": \"get_current_weather\",\n",
    "        \"description\": \"Get the current weather in a given location\",\n",
    "        \"parameters\": {\n",
    "            \"type\": \"object\",\n",
    "            \"properties\": {\n",
    "                \"location\": {\n",
    "                    \"type\": \"string\",\n",
    "                    \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
    "                },\n",
    "                \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n",
    "            },\n",
    "            \"required\": [\"location\"]\n",
    "        }\n",
    "    }\n",
    "], get_current_weather=get_current_weather)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calling the function outside of Guidance\n",
    "\n",
    "In the example above the function call was made during the execution of the guidance program, but we can also pause the program's execution whenever we want to make a function call, and then make that call outside of Guidance. This is useful if you don't want Guidance to own the function calling part of your program logic."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div id=\"guidance-stop-button-03a08751-53b3-4fa1-8dd6-b2d3890f6482\" style=\"cursor: pointer; margin: 0px; display: none; float: right; padding: 3px; border-radius: 4px 4px 4px 4px; border: 0px solid rgba(127, 127, 127, 1); padding-left: 10px; padding-right: 10px; font-size: 13px; background-color: rgba(127, 127, 127, 0.25);\">Stop program</div><div id=\"guidance-content-03a08751-53b3-4fa1-8dd6-b2d3890f6482\"><pre style='margin: 0px; padding: 0px; padding-left: 8px; margin-left: -8px; border-radius: 0px; border-left: 1px solid rgba(127, 127, 127, 0.2); white-space: pre-wrap; font-family: ColfaxAI, Arial; font-size: 15px; line-height: 23px;'><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>system</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'>You are a helpful assistant.\n",
       "\n",
       "# Tools\n",
       "\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if len(functions) &gt; 0~}}\n",
       "## functions\n",
       "\n",
       "namespace functions {\n",
       "\n",
       "{{#each functions item_name=&quot;function&quot;~}}\n",
       "// {{function.description}}\n",
       "type {{function.name}} = (_: {\n",
       "{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}\n",
       "}) =&gt; any;\n",
       "\n",
       "{{/each~}}\n",
       "} // namespace functions\n",
       "{{~/if~}}'>## functions\n",
       "\n",
       "namespace functions {\n",
       "\n",
       "<span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{#each functions item_name=&quot;function&quot;~}}\n",
       "// {{function.description}}\n",
       "type {{function.name}} = (_: {\n",
       "{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}\n",
       "}) =&gt; any;\n",
       "\n",
       "{{/each~}}\n",
       "'>// <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{function.description}}'>Get the current weather in a given location</span>\n",
       "type <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{function.name}}'>get_current_weather</span> = (_: <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{<span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}'></span>\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "'>// <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this.description}}'>The city and state, e.g. San Francisco, CA</span>\n",
       "</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{@key}}'>location</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless contains(function.parameters.required, @key)}}?{{/unless}}'></span>: <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}'><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this.type}}'>string</span></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}},{{/unless}}'>,</span>\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "'></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{@key}}'>unit</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless contains(function.parameters.required, @key)}}?{{/unless}}'>?</span>: <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}'><span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}'>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this}}'>celsius</span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}} | {{/unless}}'> | </span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this}}'>fahrenheit</span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}} | {{/unless}}'></span></span></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}},{{/unless}}'></span></span>\n",
       "}) =&gt; any;\n",
       "\n",
       "</span>} // namespace functions</span></div></div><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>user</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'>Get the current weather in New York City.</div></div><span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{~#each range(10)~}}\n",
       "    {{~#assistant~}}\n",
       "    {{gen &#x27;answer&#x27; temperature=1.0 max_tokens=50 function_call=&quot;auto&quot;}}\n",
       "    {{~/assistant~}}\n",
       "\n",
       "    {{set &#x27;function_call&#x27; extract_function_call(answer)}}\n",
       "\n",
       "    {{~#if not function_call}}{{break}}{{/if~}}\n",
       "\n",
       "    {{set &#x27;answer&#x27; await(&#x27;call_result&#x27;)}}\n",
       "\n",
       "    {{~#function name=function_call.__name__~}}\n",
       "    {{answer}}\n",
       "    {{~/function~}}\n",
       "{{~/each~}}'><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>assistant</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'><span style='background-color: rgba(0, 165, 0, 0.25); opacity: 1.0; display: inline;' title='{{gen &#x27;answer&#x27; temperature=1.0 max_tokens=50 function_call=&quot;auto&quot;}}'>\n",
       "```typescript\n",
       "functions.get_current_weather({\n",
       "  &quot;location&quot;: &quot;New York City&quot;\n",
       "})```</span></div></div><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{set &#x27;function_call&#x27; extract_function_call(answer)}}'></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{~#if not function_call}}{{break}}{{/if~}}\n",
       "\n",
       "    '></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{set &#x27;answer&#x27; await(&#x27;call_result&#x27;)}}'></span><span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{set &#x27;answer&#x27; await(&#x27;call_result&#x27;)}}</span><span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{~#function name=function_call.__name__~}}</span>\n",
       "    <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{answer}}</span>\n",
       "    <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{~/function~}}</span>\n",
       "<span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{~#each range(10) start_index=1~}}</span>\n",
       "    <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{~#assistant~}}</span>\n",
       "    <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{gen &#x27;answer&#x27; temperature=1.0 max_tokens=50 function_call=&quot;auto&quot;}}</span>\n",
       "    <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{~/assistant~}}</span>\n",
       "\n",
       "    <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{set &#x27;function_call&#x27; extract_function_call(answer)}}</span>\n",
       "\n",
       "    <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{~#if not function_call}}</span><span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{break}}</span><span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{/if~}}</span>\n",
       "\n",
       "    <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{set &#x27;answer&#x27; await(&#x27;call_result&#x27;)}}</span>\n",
       "\n",
       "    <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{~#function name=function_call.__name__~}}</span>\n",
       "    <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{answer}}</span>\n",
       "    <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{~/function~}}</span>\n",
       "<span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{{~/each~}}</span></span></pre></div>\n",
       "<script type=\"text/javascript\">(()=>{var t={296:(t,e,n)=>{var i=NaN,o=\"[object Symbol]\",r=/^\\s+|\\s+$/g,a=/^[-+]0x[0-9a-f]+$/i,s=/^0b[01]+$/i,c=/^0o[0-7]+$/i,d=parseInt,u=\"object\"==typeof n.g&&n.g&&n.g.Object===Object&&n.g,l=\"object\"==typeof self&&self&&self.Object===Object&&self,f=u||l||Function(\"return this\")(),h=Object.prototype.toString,p=Math.max,m=Math.min,g=function(){return f.Date.now()};function b(t){var e=typeof t;return!!t&&(\"object\"==e||\"function\"==e)}function y(t){if(\"number\"==typeof t)return t;if(function(t){return\"symbol\"==typeof t||function(t){return!!t&&\"object\"==typeof t}(t)&&h.call(t)==o}(t))return i;if(b(t)){var e=\"function\"==typeof t.valueOf?t.valueOf():t;t=b(e)?e+\"\":e}if(\"string\"!=typeof t)return 0===t?t:+t;t=t.replace(r,\"\");var n=s.test(t);return n||c.test(t)?d(t.slice(2),n?2:8):a.test(t)?i:+t}t.exports=function(t,e,n){var i,o,r,a,s,c,d=0,u=!1,l=!1,f=!0;if(\"function\"!=typeof t)throw new TypeError(\"Expected a function\");function h(e){var n=i,r=o;return i=o=void 0,d=e,a=t.apply(r,n)}function v(t){var n=t-c;return void 0===c||n>=e||n<0||l&&t-d>=r}function _(){var t=g();if(v(t))return w(t);s=setTimeout(_,function(t){var n=e-(t-c);return l?m(n,r-(t-d)):n}(t))}function w(t){return s=void 0,f&&i?h(t):(i=o=void 0,a)}function j(){var t=g(),n=v(t);if(i=arguments,o=this,c=t,n){if(void 0===s)return function(t){return d=t,s=setTimeout(_,e),u?h(t):a}(c);if(l)return s=setTimeout(_,e),h(c)}return void 0===s&&(s=setTimeout(_,e)),a}return e=y(e)||0,b(n)&&(u=!!n.leading,r=(l=\"maxWait\"in n)?p(y(n.maxWait)||0,e):r,f=\"trailing\"in n?!!n.trailing:f),j.cancel=function(){void 0!==s&&clearTimeout(s),d=0,i=c=o=s=void 0},j.flush=function(){return void 0===s?a:w(g())},j}},777:t=>{var e,n,i=Math.max,o=(e=function(t,e){return function(t,e,n){if(\"function\"!=typeof t)throw new TypeError(\"Expected a function\");return setTimeout((function(){t.apply(void 0,n)}),1)}(t,0,e)},n=i(void 0===n?e.length-1:n,0),function(){for(var t=arguments,o=-1,r=i(t.length-n,0),a=Array(r);++o<r;)a[o]=t[n+o];o=-1;for(var s=Array(n+1);++o<n;)s[o]=t[o];return s[n]=a,function(t,e,n){switch(n.length){case 0:return t.call(e);case 1:return t.call(e,n[0]);case 2:return t.call(e,n[0],n[1]);case 3:return t.call(e,n[0],n[1],n[2])}return t.apply(e,n)}(e,this,s)});t.exports=o}},e={};function n(i){var o=e[i];if(void 0!==o)return o.exports;var r=e[i]={exports:{}};return t[i](r,r.exports,n),r.exports}n.n=t=>{var e=t&&t.__esModule?()=>t.default:()=>t;return n.d(e,{a:e}),e},n.d=(t,e)=>{for(var i in e)n.o(e,i)&&!n.o(t,i)&&Object.defineProperty(t,i,{enumerable:!0,get:e[i]})},n.g=function(){if(\"object\"==typeof globalThis)return globalThis;try{return this||new Function(\"return this\")()}catch(t){if(\"object\"==typeof window)return window}}(),n.o=(t,e)=>Object.prototype.hasOwnProperty.call(t,e),(()=>{\"use strict\";const t=t=>{const e=new Set;do{for(const n of Reflect.ownKeys(t))e.add([t,n])}while((t=Reflect.getPrototypeOf(t))&&t!==Object.prototype);return e};function e(e,{include:n,exclude:i}={}){const o=t=>{const e=e=>\"string\"==typeof e?t===e:e.test(t);return n?n.some(e):!i||!i.some(e)};for(const[n,i]of t(e.constructor.prototype)){if(\"constructor\"===i||!o(i))continue;const t=Reflect.getOwnPropertyDescriptor(n,i);t&&\"function\"==typeof t.value&&(e[i]=e[i].bind(e))}return e}var i=n(777),o=n.n(i),r=n(296),a=n.n(r);class s{constructor(t,n){e(this),this.interfaceId=t,this.callbackMap={},this.data={},this.pendingData={},this.jcomm=new c(\"guidance_interface_target_\"+this.interfaceId,this.updateData,\"open\"),this.debouncedSendPendingData500=a()(this.sendPendingData,500),this.debouncedSendPendingData1000=a()(this.sendPendingData,1e3),n&&o()(n)}send(t,e){this.addPendingData(t,e),this.sendPendingData()}sendEvent(t){for(const e of Object.keys(t))this.addPendingData(e,t[e]);this.sendPendingData()}debouncedSendEvent500(t){for(const e of Object.keys(t))this.addPendingData(e,t[e]);this.debouncedSendPendingData500()}debouncedSend500(t,e){this.addPendingData(t,e),this.debouncedSendPendingData500()}debouncedSend1000(t,e){this.addPendingData(t,e),this.debouncedSendPendingData1000()}addPendingData(t,e){Array.isArray(t)||(t=[t]);for(const n in t)this.pendingData[t[n]]=e}updateData(t){t=JSON.parse(t.data);for(const e in t)this.data[e]=t[e];for(const e in t)e in this.callbackMap&&this.callbackMap[e](this.data[e])}subscribe(t,e){this.callbackMap[t]=e,o()((e=>this.callbackMap[t](this.data[t])))}sendPendingData(){this.jcomm.send_data(this.pendingData),this.pendingData={}}}class c{constructor(t,e,n=\"open\"){this._fire_callback=this._fire_callback.bind(this),this._register=this._register.bind(this),this.jcomm=void 0,this.callback=e,void 0!==window.Jupyter?\"register\"===n?Jupyter.notebook.kernel.comm_manager.register_target(t,this._register):(this.jcomm=Jupyter.notebook.kernel.comm_manager.new_comm(t),this.jcomm.on_msg(this._fire_callback)):void 0!==window._mgr&&(\"register\"===n?window._mgr.widgetManager.proxyKernel.registerCommTarget(t,this._register):(this.jcomm=window._mgr.widgetManager.proxyKernel.createComm(t),this.jcomm.open({},\"\"),this.jcomm.onMsg=this._fire_callback))}send_data(t){void 0!==this.jcomm?this.jcomm.send(t):console.error(\"Jupyter comm module not yet loaded! So we can't send the message.\")}_register(t,e){this.jcomm=t,this.jcomm.on_msg(this._fire_callback)}_fire_callback(t){this.callback(t.content.data)}}class d{constructor(t,n){e(this),this.id=t,this.comm=new s(t),this.comm.subscribe(\"append\",this.appendData),this.comm.subscribe(\"replace\",this.replaceData),this.comm.subscribe(\"event\",this.eventOccurred),this.element=document.getElementById(\"guidance-content-\"+t),this.stop_button=document.getElementById(\"guidance-stop-button-\"+t),this.stop_button.onclick=()=>this.comm.send(\"event\",\"stop\")}appendData(t){t&&(this.stop_button.style.display=\"inline-block\",this.element.innerHTML+=t)}replaceData(t){t&&(this.stop_button.style.display=\"inline-block\",this.element.innerHTML=t)}eventOccurred(t){\"complete\"===t&&(this.stop_button.style.display=\"none\")}}window._guidanceDisplay=function(t,e){return new d(t,e)}})()})();; window._guidanceDisplay(\"03a08751-53b3-4fa1-8dd6-b2d3890f6482\");</script>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# define a guidance program that pauses we when a function call is made\n",
    "await_program = guidance(\"\"\"\n",
    "{{~#system~}}\n",
    "You are a helpful assistant.\n",
    "{{>tool_def functions=functions}}\n",
    "{{~/system~}}\n",
    "\n",
    "{{~#user~}}\n",
    "Get the current weather in New York City.\n",
    "{{~/user~}}\n",
    "\n",
    "{{~#each range(10)~}}\n",
    "    {{~#assistant~}}\n",
    "    {{gen 'answer' temperature=1.0 max_tokens=50 function_call=\"auto\"}}\n",
    "    {{~/assistant~}}\n",
    "\n",
    "    {{set 'function_call' extract_function_call(answer)}}\n",
    "\n",
    "    {{~#if not function_call}}{{break}}{{/if~}}\n",
    "\n",
    "    {{set 'answer' await('call_result')}}\n",
    "\n",
    "    {{~#function name=function_call.__name__~}}\n",
    "    {{answer}}\n",
    "    {{~/function~}}\n",
    "{{~/each~}}\"\"\")\n",
    "\n",
    "# call the program, passing in the function definition we want to use as JSON\n",
    "executed_await_program = await_program(functions=[\n",
    "    {\n",
    "        \"name\": \"get_current_weather\",\n",
    "        \"description\": \"Get the current weather in a given location\",\n",
    "        \"parameters\": {\n",
    "            \"type\": \"object\",\n",
    "            \"properties\": {\n",
    "                \"location\": {\n",
    "                    \"type\": \"string\",\n",
    "                    \"description\": \"The city and state, e.g. San Francisco, CA\",\n",
    "                },\n",
    "                \"unit\": {\"type\": \"string\", \"enum\": [\"celsius\", \"fahrenheit\"]},\n",
    "            },\n",
    "            \"required\": [\"location\"],\n",
    "        }\n",
    "    }\n",
    "], get_current_weather=get_current_weather)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CallableAnswer(__name__=get_current_weather, __kwdefaults__={'location': 'New York City'})"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# these are the details of the function call we need to make\n",
    "executed_await_program[\"function_call\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div id=\"guidance-stop-button-975f4db7-4bf5-4d37-a1cc-31fef055348d\" style=\"cursor: pointer; margin: 0px; display: none; float: right; padding: 3px; border-radius: 4px 4px 4px 4px; border: 0px solid rgba(127, 127, 127, 1); padding-left: 10px; padding-right: 10px; font-size: 13px; background-color: rgba(127, 127, 127, 0.25);\">Stop program</div><div id=\"guidance-content-975f4db7-4bf5-4d37-a1cc-31fef055348d\"><pre style='margin: 0px; padding: 0px; padding-left: 8px; margin-left: -8px; border-radius: 0px; border-left: 1px solid rgba(127, 127, 127, 0.2); white-space: pre-wrap; font-family: ColfaxAI, Arial; font-size: 15px; line-height: 23px;'><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>system</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'>You are a helpful assistant.\n",
       "\n",
       "# Tools\n",
       "\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if len(functions) &gt; 0~}}\n",
       "## functions\n",
       "\n",
       "namespace functions {\n",
       "\n",
       "{{#each functions item_name=&quot;function&quot;~}}\n",
       "// {{function.description}}\n",
       "type {{function.name}} = (_: {\n",
       "{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}\n",
       "}) =&gt; any;\n",
       "\n",
       "{{/each~}}\n",
       "} // namespace functions\n",
       "{{~/if~}}'>## functions\n",
       "\n",
       "namespace functions {\n",
       "\n",
       "<span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{#each functions item_name=&quot;function&quot;~}}\n",
       "// {{function.description}}\n",
       "type {{function.name}} = (_: {\n",
       "{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}\n",
       "}) =&gt; any;\n",
       "\n",
       "{{/each~}}\n",
       "'>// <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{function.description}}'>Get the current weather in a given location</span>\n",
       "type <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{function.name}}'>get_current_weather</span> = (_: <span style='font-family: monospace; background-color: rgba(0, 0, 0, 0.05);'>{<span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{~#each function.parameters.properties}}\n",
       "{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "{{@key}}{{#unless contains(function.parameters.required, @key)}}?{{/unless}}: {{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}{{#unless @last}},{{/unless}}\n",
       "{{~/each}}'></span>\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "'>// <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this.description}}'>The city and state, e.g. San Francisco, CA</span>\n",
       "</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{@key}}'>location</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless contains(function.parameters.required, @key)}}?{{/unless}}'></span>: <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}'><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this.type}}'>string</span></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}},{{/unless}}'>,</span>\n",
       "<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;description&quot;)}}// {{this.description}}\n",
       "{{/if~}}\n",
       "'></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{@key}}'>unit</span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless contains(function.parameters.required, @key)}}?{{/unless}}'>?</span>: <span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#if contains(this, &quot;enum&quot;)}}{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}{{else}}{{this.type}}{{/if}}'><span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{#each this.enum}}&quot;{{this}}&quot;{{#unless @last}} | {{/unless}}{{/each}}'>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this}}'>celsius</span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}} | {{/unless}}'> | </span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{this}}'>fahrenheit</span>&quot;<span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}} | {{/unless}}'></span></span></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{#unless @last}},{{/unless}}'></span></span>\n",
       "}) =&gt; any;\n",
       "\n",
       "</span>} // namespace functions</span></div></div><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>user</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'>Get the current weather in New York City.</div></div><span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{~#each range(10)~}}\n",
       "    {{~#assistant~}}\n",
       "    {{gen &#x27;answer&#x27; temperature=1.0 max_tokens=50 function_call=&quot;auto&quot;}}\n",
       "    {{~/assistant~}}\n",
       "\n",
       "    {{set &#x27;function_call&#x27; extract_function_call(answer)}}\n",
       "\n",
       "    {{~#if not function_call}}{{break}}{{/if~}}\n",
       "\n",
       "    {{set &#x27;answer&#x27; await(&#x27;call_result&#x27;)}}\n",
       "\n",
       "    {{~#function name=function_call.__name__~}}\n",
       "    {{answer}}\n",
       "    {{~/function~}}\n",
       "{{~/each~}}'><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>assistant</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'><span style='background-color: rgba(0, 165, 0, 0.25); opacity: 1.0; display: inline;' title='{{gen &#x27;answer&#x27; temperature=1.0 max_tokens=50 function_call=&quot;auto&quot;}}'>\n",
       "```typescript\n",
       "functions.get_current_weather({\n",
       "  &quot;location&quot;: &quot;New York City&quot;\n",
       "})```</span></div></div><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{set &#x27;function_call&#x27; extract_function_call(answer)}}'></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{~#if not function_call}}{{break}}{{/if~}}\n",
       "\n",
       "    '></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{set &#x27;answer&#x27; await(&#x27;call_result&#x27;)}}'></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{set &#x27;answer&#x27; await(&#x27;call_result&#x27;)}}'></span><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>function</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{answer}}'>{&quot;location&quot;: &quot;New York City&quot;, &quot;temperature&quot;: &quot;71&quot;, &quot;unit&quot;: &quot;fahrenheit&quot;, &quot;forecast&quot;: [&quot;sunny&quot;, &quot;windy&quot;]}</span></div></div><span style='opacity: 1.0; display: inline; background-color: rgba(165, 165, 165, 0.1);' title='{{~#each range(10) start_index=1~}}\n",
       "    {{~#assistant~}}\n",
       "    {{gen &#x27;answer&#x27; temperature=1.0 max_tokens=50 function_call=&quot;auto&quot;}}\n",
       "    {{~/assistant~}}\n",
       "\n",
       "    {{set &#x27;function_call&#x27; extract_function_call(answer)}}\n",
       "\n",
       "    {{~#if not function_call}}{{break}}{{/if~}}\n",
       "\n",
       "    {{set &#x27;answer&#x27; await(&#x27;call_result&#x27;)}}\n",
       "\n",
       "    {{~#function name=function_call.__name__~}}\n",
       "    {{answer}}\n",
       "    {{~/function~}}\n",
       "{{~/each~}}'><div style='display: flex; border-bottom: 1px solid rgba(127, 127, 127, 0.2); align-items: center;'><div style='flex: 0 0 80px; opacity: 0.5;'>assistant</div><div style='flex-grow: 1; padding: 5px; padding-top: 10px; padding-bottom: 10px; margin-top: 0px; white-space: pre-wrap; margin-bottom: 0px;'><span style='background-color: rgba(0, 165, 0, 0.25); opacity: 1.0; display: inline;' title='{{gen &#x27;answer&#x27; temperature=1.0 max_tokens=50 function_call=&quot;auto&quot;}}'>The current weather in New York City is 71°F and it is sunny and windy.</span></div></div><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{set &#x27;function_call&#x27; extract_function_call(answer)}}'></span><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{~#if not function_call}}{{break}}{{/if~}}\n",
       "\n",
       "    '><span style='background-color: rgba(0, 138.56128016, 250.76166089, 0.25); display: inline;' title='{{break}}'></span></span></span></span></pre></div>\n",
       "<script type=\"text/javascript\">(()=>{var t={296:(t,e,n)=>{var i=NaN,o=\"[object Symbol]\",r=/^\\s+|\\s+$/g,a=/^[-+]0x[0-9a-f]+$/i,s=/^0b[01]+$/i,c=/^0o[0-7]+$/i,d=parseInt,u=\"object\"==typeof n.g&&n.g&&n.g.Object===Object&&n.g,l=\"object\"==typeof self&&self&&self.Object===Object&&self,f=u||l||Function(\"return this\")(),h=Object.prototype.toString,p=Math.max,m=Math.min,g=function(){return f.Date.now()};function b(t){var e=typeof t;return!!t&&(\"object\"==e||\"function\"==e)}function y(t){if(\"number\"==typeof t)return t;if(function(t){return\"symbol\"==typeof t||function(t){return!!t&&\"object\"==typeof t}(t)&&h.call(t)==o}(t))return i;if(b(t)){var e=\"function\"==typeof t.valueOf?t.valueOf():t;t=b(e)?e+\"\":e}if(\"string\"!=typeof t)return 0===t?t:+t;t=t.replace(r,\"\");var n=s.test(t);return n||c.test(t)?d(t.slice(2),n?2:8):a.test(t)?i:+t}t.exports=function(t,e,n){var i,o,r,a,s,c,d=0,u=!1,l=!1,f=!0;if(\"function\"!=typeof t)throw new TypeError(\"Expected a function\");function h(e){var n=i,r=o;return i=o=void 0,d=e,a=t.apply(r,n)}function v(t){var n=t-c;return void 0===c||n>=e||n<0||l&&t-d>=r}function _(){var t=g();if(v(t))return w(t);s=setTimeout(_,function(t){var n=e-(t-c);return l?m(n,r-(t-d)):n}(t))}function w(t){return s=void 0,f&&i?h(t):(i=o=void 0,a)}function j(){var t=g(),n=v(t);if(i=arguments,o=this,c=t,n){if(void 0===s)return function(t){return d=t,s=setTimeout(_,e),u?h(t):a}(c);if(l)return s=setTimeout(_,e),h(c)}return void 0===s&&(s=setTimeout(_,e)),a}return e=y(e)||0,b(n)&&(u=!!n.leading,r=(l=\"maxWait\"in n)?p(y(n.maxWait)||0,e):r,f=\"trailing\"in n?!!n.trailing:f),j.cancel=function(){void 0!==s&&clearTimeout(s),d=0,i=c=o=s=void 0},j.flush=function(){return void 0===s?a:w(g())},j}},777:t=>{var e,n,i=Math.max,o=(e=function(t,e){return function(t,e,n){if(\"function\"!=typeof t)throw new TypeError(\"Expected a function\");return setTimeout((function(){t.apply(void 0,n)}),1)}(t,0,e)},n=i(void 0===n?e.length-1:n,0),function(){for(var t=arguments,o=-1,r=i(t.length-n,0),a=Array(r);++o<r;)a[o]=t[n+o];o=-1;for(var s=Array(n+1);++o<n;)s[o]=t[o];return s[n]=a,function(t,e,n){switch(n.length){case 0:return t.call(e);case 1:return t.call(e,n[0]);case 2:return t.call(e,n[0],n[1]);case 3:return t.call(e,n[0],n[1],n[2])}return t.apply(e,n)}(e,this,s)});t.exports=o}},e={};function n(i){var o=e[i];if(void 0!==o)return o.exports;var r=e[i]={exports:{}};return t[i](r,r.exports,n),r.exports}n.n=t=>{var e=t&&t.__esModule?()=>t.default:()=>t;return n.d(e,{a:e}),e},n.d=(t,e)=>{for(var i in e)n.o(e,i)&&!n.o(t,i)&&Object.defineProperty(t,i,{enumerable:!0,get:e[i]})},n.g=function(){if(\"object\"==typeof globalThis)return globalThis;try{return this||new Function(\"return this\")()}catch(t){if(\"object\"==typeof window)return window}}(),n.o=(t,e)=>Object.prototype.hasOwnProperty.call(t,e),(()=>{\"use strict\";const t=t=>{const e=new Set;do{for(const n of Reflect.ownKeys(t))e.add([t,n])}while((t=Reflect.getPrototypeOf(t))&&t!==Object.prototype);return e};function e(e,{include:n,exclude:i}={}){const o=t=>{const e=e=>\"string\"==typeof e?t===e:e.test(t);return n?n.some(e):!i||!i.some(e)};for(const[n,i]of t(e.constructor.prototype)){if(\"constructor\"===i||!o(i))continue;const t=Reflect.getOwnPropertyDescriptor(n,i);t&&\"function\"==typeof t.value&&(e[i]=e[i].bind(e))}return e}var i=n(777),o=n.n(i),r=n(296),a=n.n(r);class s{constructor(t,n){e(this),this.interfaceId=t,this.callbackMap={},this.data={},this.pendingData={},this.jcomm=new c(\"guidance_interface_target_\"+this.interfaceId,this.updateData,\"open\"),this.debouncedSendPendingData500=a()(this.sendPendingData,500),this.debouncedSendPendingData1000=a()(this.sendPendingData,1e3),n&&o()(n)}send(t,e){this.addPendingData(t,e),this.sendPendingData()}sendEvent(t){for(const e of Object.keys(t))this.addPendingData(e,t[e]);this.sendPendingData()}debouncedSendEvent500(t){for(const e of Object.keys(t))this.addPendingData(e,t[e]);this.debouncedSendPendingData500()}debouncedSend500(t,e){this.addPendingData(t,e),this.debouncedSendPendingData500()}debouncedSend1000(t,e){this.addPendingData(t,e),this.debouncedSendPendingData1000()}addPendingData(t,e){Array.isArray(t)||(t=[t]);for(const n in t)this.pendingData[t[n]]=e}updateData(t){t=JSON.parse(t.data);for(const e in t)this.data[e]=t[e];for(const e in t)e in this.callbackMap&&this.callbackMap[e](this.data[e])}subscribe(t,e){this.callbackMap[t]=e,o()((e=>this.callbackMap[t](this.data[t])))}sendPendingData(){this.jcomm.send_data(this.pendingData),this.pendingData={}}}class c{constructor(t,e,n=\"open\"){this._fire_callback=this._fire_callback.bind(this),this._register=this._register.bind(this),this.jcomm=void 0,this.callback=e,void 0!==window.Jupyter?\"register\"===n?Jupyter.notebook.kernel.comm_manager.register_target(t,this._register):(this.jcomm=Jupyter.notebook.kernel.comm_manager.new_comm(t),this.jcomm.on_msg(this._fire_callback)):void 0!==window._mgr&&(\"register\"===n?window._mgr.widgetManager.proxyKernel.registerCommTarget(t,this._register):(this.jcomm=window._mgr.widgetManager.proxyKernel.createComm(t),this.jcomm.open({},\"\"),this.jcomm.onMsg=this._fire_callback))}send_data(t){void 0!==this.jcomm?this.jcomm.send(t):console.error(\"Jupyter comm module not yet loaded! So we can't send the message.\")}_register(t,e){this.jcomm=t,this.jcomm.on_msg(this._fire_callback)}_fire_callback(t){this.callback(t.content.data)}}class d{constructor(t,n){e(this),this.id=t,this.comm=new s(t),this.comm.subscribe(\"append\",this.appendData),this.comm.subscribe(\"replace\",this.replaceData),this.comm.subscribe(\"event\",this.eventOccurred),this.element=document.getElementById(\"guidance-content-\"+t),this.stop_button=document.getElementById(\"guidance-stop-button-\"+t),this.stop_button.onclick=()=>this.comm.send(\"event\",\"stop\")}appendData(t){t&&(this.stop_button.style.display=\"inline-block\",this.element.innerHTML+=t)}replaceData(t){t&&(this.stop_button.style.display=\"inline-block\",this.element.innerHTML=t)}eventOccurred(t){\"complete\"===t&&(this.stop_button.style.display=\"none\")}}window._guidanceDisplay=function(t,e){return new d(t,e)}})()})();; window._guidanceDisplay(\"975f4db7-4bf5-4d37-a1cc-31fef055348d\");</script>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# run the call\n",
    "call = executed_await_program[\"function_call\"]\n",
    "if call.__name__ == \"get_current_weather\":\n",
    "    weather = get_current_weather(**call.__kwdefaults__)\n",
    "\n",
    "executed_await_program(call_result=weather)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Open Source model [TODO]\n",
    "\n",
    "Here we run the same examples as before, but with the an open model instead. Note that the model does not have any special fine-tuned support for function calls, so we have to provide much more detail in the tool definition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<hr style=\"height: 1px; opacity: 0.5; border: none; background: #cccccc;\">\n",
    "<div style=\"text-align: center; opacity: 0.5\">Have an idea for more helpful examples? Pull requests that add to this documentation notebook are encouraged!</div>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "base",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
